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Vector Precoding for Wireless MIMO Systems: 

A Replica Analysis 

Ralf R. Miiller, Dongning Guo, and Aris L. Moustakas 



Abstract 

We apply the replica method to analyze vector precoding, a method to reduce transmit power in antenna array 

communications. The analysis applies to a very general class of channel matrices. The statistics of the channel 

matrix enter the transmitted energy per symbol via its R-transform. We find that vector precoding performs much 

' better for complex than for real alphabets. 

As a byproduct, we find a nonlinear precoding method with polynomial complexity that outperforms NP-hard 

Tomlinson-Harashima precoding for binary modulation on complex channels if the number of transmit antennas is 

Ch ! slightly larger than twice the number of receive antennas. 
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I. Introduction 

Wireless multiple-input multiple-output (MIMO) systems offer the possibility to increase data rate 
over conventional wireless communications without need for more physical radio spectrum by means of 
multiple antenna elements at both transmitter and receiver side. Since the pioneering work in the field [5], 
[18], countless implementations for those MIMO systems have been proposed. They can be classified by 
the side where the signal processing takes place. Depending on the proposed system solution, there can 
■ be need for major signal processing at the receiver side, the transmitter side or both of them. This work 
G\ ', is concerned with systems where sophisticated signal processing is required solely at the transmitter side. 
^ ; This is advantageous for transmitting data to low-cost or battery-driven devices such as cell-phones and 
t-h ■ PDAs. 

\q . It is an unavoidable feature of wireless MIMO systems that signals sent at different antenna elements 
of the transmit array are received with severe crosstalk at the respective antenna elements of the receive 

^ ; array. In order to compensate for this crosstalk, one can use linear joint transmitter processing, also known 
as linear vector precoding, as suggested in [22], [16]. This comes, however, at the expense of the need 

• »-j . for an increased transmit power in order to maintain the distance properties of the signal constellation. A 
more sophisticated method for transmitter processing is nonlinear vector precoding, in this work simply 
c3 ' referred to as vector precoding. It is based on the concept of Tomlinson-Harashima preecoding [19], [7] 
which was originally proposed to combat intersymbol interference. It was proposed for use in context of 
MIMO systems in [23], [9]. For a general survey on vector precoding the reader is referred to [4]. 

In this work, we are mainly concerned with the performance analysis of vector precoding. To the best of 
our knowledge, there is no published literature on the performance analysis of nonlinear vector precoding 
by analytical means. This paper aims to pave the way a first step forward towards this direction employing 
the replica method which was originally invented for the analysis of spin glasses in statistical physics [13], 
[3] and has become increasingly powerful to address problems in wireless communications and coding 
theory [14]. We use the analytical results developed in this paper to compare real- valued vector precoding 
with complex- valued vector precoding as well as with some hybrid forms of it which are newly proposed 
in this work. 
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The paper is composed of five more sections. Section [TT] introduces vector precoding from a general 
point of view. This point of view is more general than the way vector precoding is dealt with in 
the references mentioned earlier, but it is well suited to the replica analysis to follow. Section HIT] 
formulates vector precoding as a non-convex quadratic programming problem and introduces the technical 
assumptions that we require for the analytical analysis. Section ITVl derives the general replica symmetric 
solution to any non-convex quadratic programming problem for which the search space can be factorized 
into Kronecker products of scalar sets in the limit of a large number of dimensions of the search 
space. Section |V] specializes the general results to MIMO channels with channel matrices composed of 
independent identically distributed entries and various choices for the relaxation of the symbol alphabet. 
Section [VI] summarizes the main conclusions. Particularly technical derivations are placed in the two 
appendices. 



II. Vector Precoding 

Vector precoding aims to minimize the transmitted power that is associated with the transmission of 
a certain data vector s e S K of length K. For that purpose, the original symbol alphabet S is relaxed 
into the alphabet B. The data representation in the relaxed alphabet is redundant. That means that several 
symbols in the relaxed alphabet represent the same data. Due to the redundant representation, we can 
now choose that representation of our data which requires the least power to be transmitted. This way of 
saving transmit power is what we call vector precoding. 

That means, for any s £ S there is a set B s C B such that all elements of B s represent the data s. 
Take binary transmission as an example, i.e. S = {1,0}. Without vector precoding, it is most common 
to choose £>o = {+1} and B\ = { — 1}. This modulation is called binary phase shift keying. For binary 
modulation, vector precoding is the idea to have B D {+1} and B\ D { — 1}, i-e- to allow for supersets 
of the binary constellation. A popular choice for those supersets is due to Tomlinson and Harashima [19], 
[7], see also Fig. [TJ Here, we have B = 4Z + 1 and B\ = 4Z — 1. 

In order to avoid ambiguities, we should have 

B i nB j = ® V^j. (1) 

In addition, one would like to design the sets B. t such that the distance properties between the presented 
information are preserved. This is easily achieved by letting the sets Bi to be distinct sub-lattices of 
B. However, we are not concerned with these design issues here. We aim to analyze the power saving 
achieved by a particular choice of the sets £>j. This goal is achieved using the replica method invented in 
statistical physics. 



III. Problem Statement 

Let s = [si, . . . , sk] t denote the information to be encoded. Let t = Tx be the vector being sent. 
Then, the precoding problem can be written as the minimization of the following quadratic form 

min 1 1 Tx 1 1 2 = min x' Jx (2) 

over the discrete set 

X = B S1 x B S2 x • • • x B SK (3) 

with J = T^T. This type of problem is known in context of optimization as non-convex quadratic 
programming. 

In order to allow for analytical tractability, we need the follwing definition and assumptions: 
Definition 1 (R-TRANSFORM) Let P(x) denote an arbitrary probability distribution. Let 

m{s) (4) 
/ x - s 
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Then, the R-transform of P(x) is 



R{w) 



in 



-w) 



1 

w 



(5) 



with m 1 (w) denoting the inverse of m(s) with respect to composition, i.e. m 1 (m(s)) = s. 



Assumption 1 (self-averaging property) We have 



lim Pr ( — 

K^oo \ K 



min x^Jx — E min x^Jx 

x£X j x£X 



> £ 







(6) 



for all e > 0, i.e. convergence in probability. 

Assumption 2 (replica continuity) For all (3 > 0, the continuation of the function 



fin) 



a=l x a £X 



(7) 



onto the positive real line is equal to 



in the right-sided vicinity of n = 0. 



-fix^Jx 



Assumption 3 (unitary invariance) The random matrix J, can be decomposed into 

J = ODO ] 



(8) 



such that the matrices D and O are diagonal and Haar distributed, respectively. Moreover, as K — ► oo, 
the asymptotic eigenvalue distribution of J converges to a non-random distribution function which can 
be uniquely characterized by its R-transform R(w). 

ASSUMPTION 4 (replica symmetry) When applying the replica method to solve the saddle-point equa- 
tions, we will assume that the extremal point is invariant to permutations of the replica index. For a detailed 
discussion of replica symmetry, the reader is referred to the literature of spin glasses, e.g. [13], [3]. 

The first three assumptions are rather technical and should hold well in the application we are addressing. 
The validity of replica symmetry for the minimum value of x^Jx is an approximation, which is made 
for analytical tractability and, for sake of simplicity, no further justification is made here. It should be 
pointed out though that, even when replica symmetry is not valid the correct value of quantities such as 
E s do not differ much from the corresponding values evaluated within the replica- symmetric assumption, 
cf. [11], [12]. 

IV. General Result 

In this section, we derive a general solution to the non-convex quadratic programming problem © in 
the limit of a large number of dimensions K. We find the following result: 

Proposition 1 Let Assumptions 1 to 4 hold. Let P s (s) denote the limit of the empirical distribution 
function of the information symbols Si,...,sk as K — > oo. Moreover, let the parameters q and b be 
solutions to the following pair of coupled fixed-point equations 



argmm 



3? ^ argmin 

x&Bs 



qR'(-b) 
2R 2 (-b) 



x 



DzdP s (s) 



qR'(-b) 
2R 2 (-b) 



1 DzdP s (s) 
' J y/2qR'(-b) 



(9) 
(10) 
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with Dz = exp(— z 2 /2) / {2n)dz being the complex Gaussian measure. Then, if < b < oo, we have 



1 d 
— min x^Jx — > q — bR(—b) 
K x&x db 



(11) 



in probability as K — > oo. 



The remainder of this section is dedicated to the derivation of Proposition 1 . Further sections will not 
make reference to the remainder of Section |IVj 

With Assumptions 1 and 2, we find for the average transmitted energy per symbol in the large system 
limit 



E s = lim — mina^Ja: 

k^oo K xex 



- lim lim E log V e ~^ Jx 

H x£X 

lim -L lim |- log E ( V e ~^ A 

J->oo BK n->o On J J 

X d 1 n 

lim — lim — lim — logET e ~Px a jx a 

P^oo B n~*0 dn K^oo K J , 



— lim 



(12) 
(13) 

(14) 



a=l X a &X 



where the argument of the logarithm in (fl"5l) is given by3 



lim — log E 



exp 



{x a ex} 



lim — log E 



exp 



{Xa&X} 



a=l 



a=l 



(15) 



Using Assumption 3, we can integrate over the Haar distributed eigenvectors of J. This integration 
was studied by Harish-Chandra [8] and Itzykson & Zuber [10] in the mathematics and physics literature, 
respectively. It was applied in context of wireless communications in [17]. Guionnet and Mai'da [6] solve 
this integral in terms of the R-transform R(w) of the asymptotic eigenvalue distribution of J. Following 
their approach yields 



S « = i im IP lo S 52 ex P 

{Xa€X} 

with Aj denoting the n positive eigenvalues of 



n Aa 

KJ2 / R(~w)dw 

a=l "I 



(16) 



(17) 



a = l 



The eigenvalues Aj are completely determined by the inner products 



K 



KQab X a X b ^ ^ 2- , 



k x bk- 



(18) 



k=l 



'The notation ^2r x } ls used as shortcut for Y^ X1 ' ' ' 
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In order to perform the summation in (fT6l) , the A'n-dimensional space spanned by the replicas is split 
into subshells 

S{Q}={x 1 ,...,x n \xlx b = KQ ab ] (19) 

where the inner product of two different replicated vectors x a and x b is constant in each subshelll With 
this splitting of the space, we fincH 

S„= lim llog f e K ^e- KS ^T[dQ ab , (20) 

where 

${Q} = Y1 / ^(-^) d ^ (21) 

0=1 { 

and 

e Ki {Q}= ^ Y[ S (x{x b -KQ ab ) (22) 

{x a &X} a,b 

denotes the probability weight of the subshell composed of two-dimensional Dirac-functions in the 
complex plane. This procedure is a change of integration variables in multiple dimensions where the 
integration of an exponential function over the replicas has been replaced by integration over the variables 
{Q}. In the following the two exponential terms in (1201) are evaluated separately. 
First, we turn to the evaluation of the measure e KI ^ Q \ The Fourier expansion of the Dirac measure 



5 (x\x b - KQ ab ) = J exp Q ab (x\x b - KQ ab ) 



dQ 



ab 



J 



2vrj 



(23) 



with J — {t — joo; t + joo), gives 



ki{Q} = ^ Y\ J c^ ab ^ aXb ~ KQab ^ ^ ab (24) 



e 

{x a &X} a,b j 



C 



>,hM k {Qy K TQ« b Qa b &Q ab 



nif (25) 



with 



u,6 



( ~~\ yr—^ QabK X b 

M k [q| = Yl ea - b ■ w 

In the limit of K — ► oo one of the exponential terms in (|2Q|) will dominate over all others. Thus, only 
the maximum value of the correlation Q ab is relevant for calculation of the integral. 

At this point, we assume replica symmetry. This means, that in order to find the maximum of the 
objective function, we consider only a subset of the potential possibilities that the variables Q ab could 
take. In particular, we are interested in the most general form of the positive semidefinite matrix Q with 
permutational symmetry when exchanging its replica indices. Therefore, we need a matrix with all off- 
diagonal elements equal to each other. Thus, we restrict them to the following two different possibilities 
Qab = q,Va ^ b and Q aa = q + &//3,Va where b > since Q has to be positive semidefinite. One 
case distinction has been made to distinguish correlations Q ab which correspond to correlations between 

2 The notation f{Q} expresses dependency of the function /(■) on Q a (,Va, b. 
3 The notation JT b is used as shortcut for FIa=i n"=i - 
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different and identical replica indices, respectively. We apply the same idea to the correlation variables 
in the dual (Fourier) domain and set with a modest amount of foresight Q ab = /3 2 / 2 /2,Va ^ b and 
Qaa = /9 2 / 2 /2 — /3e,Va. Note that despite the fact that Q is complex, in general, its values at the 
saddle-point are in fact real. 

At this point the crucial benefit of the replica method becomes obvious. Assuming replica continuity, we 
have managed to reduce the evaluation of a continuous function to sampling it at integer points. Assuming 
replica symmetry we have reduced the task of evaluating infinitely many integer points to calculating four 
different correlations (two in the original and two in the Fourier domain). 

The assumption of replica symmetry leads to 

£ Q ab Q ab = (3 2 fq + n (if - e) {(3q + b) (27) 

a,b ^ ' 

and 

n n 

M k (eJ)= e 0=1 b=a+1 < 28) 

{x a £B Sk } 

Note that the prior distribution enters the free energy only via (128T ). We will focus on this later on after 
having finished with the other terms. 

For the evaluation of Q{Q} in (1201 ), we can use the replica symmetry to explicitly calculate the 
eigenvalues A/. Considerations of linear algebra lead to the conclusion that the eigenvalues b and b + (3nq 
occur with multiplicities n — 1 and 1, respectively. Thus we get 

b b+/3nq 

G(q,b) = (n- 1) / R(-w)dw+ [ R(-w)dw. (29) 



Since the integral in (1201) is dominated by the maximum argument of the exponential function, the 
derivatives of 

G{Q} + J2^- b W 

with respect to q and b must vanish as K — > 00 Taking derivatives after plugging (ITTT) and (1291) into 
(130b . gives 

PnR(-b-Pnq) + ^^(3 2 f + (3n(if-e] = 



{n-l)R(-b) + R{-b-Pnq)+n[—-e) = 



solving for e and / gives 



e = R(-b) (31) 

= lR { - b) -R { - b -^) 
V pn 



with the limits for n — > 



/ =3 ^2qR'{-b) (33) 

n 9 4- =5 0. (34) 
on 

4 It turns out that when lim n ^o d„E. n is expressed in terms of e, /, q, b, the relevant extremum is in fact a maximum and not a minimum. 
This is due to the fact that when drops below unity, the minima of a function become maxima and vice-versa. For a detailed analysis of this 
technicality, see [15]. 
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Consider now the integration over the prior distribution in the moment-generating function. Consider 
(|28l ) giving the only term that involves the prior distribution and apply the complex Hubbard-Stratonovich 
transform 



e 2 



2tt 



e^ {xz * } ~-^dz = / e* {xz} Bz. 



(35) 



Then, we find with d28 



M k (e,f) 



E »" 



E 

{x a eB Sfc } 



E ^ 

a = l 



- E £eM^ 
tt=l 



/3 E /K{x aZ *}-e|^'a| 2 

e a=1 Dz 



E 



^f3fX{xz*}-fle\xf 



Bz 



(36) 
(37) 

(38) 



Moreover, for K — > 00, we have by the law of large numbers 

K 



logM(e,/) = ilogf[M fc (e,/) 

fc=i 



(39) 



In the large system limit, the integral in (1251) is dominated by that value of the integration variable 
which maximizes the argument of the exponential function. Thus, partial derivatives of 



logM(e,/) 



U ( n ~ !) f 2n2 

^ / P q-n 



Pf 



-e){b + Pq) 



(40) 



with respect to / and e must vanish as K — > 00. 

An explicit calculation of the two derivatives gives the following expressions for the macroscopic 
parameters q and b 



1 



^qR\-b) 



ExeS Sft{ z * x y e P^2qR>(-b)Vt{z*x}-l3R(-b)\x\ r < 



V p /3 v /2 g 7f'(-fe)J?{ Z * : r}-/3R(-fe)|x| 5 



■DzdPis) 



^B„ Fl e D^dP(s) - - 

V p Py/2qR>(-b)3l{z*x}-l3R(-b)\x\2 * V j3 



Moreover, we find 



, db 
hm — 

n-»0 C77T, 



0. 



(41) 
(42) 

(43) 



Finally, the fixed point equations ((42)) and (HT)) simplify via the saddle point integration rule to © and 
(PTOl) . Note that the minimization with respect to the symbol x splits the integration space of z into the 
Voronoi regions defined by the (appropriately scaled) signal constellation B s . 



s 



Returning to our initial goal, the evaluation of the average transmitted energy per symbol, and collecting 
our previous results, we find 

1 d 

E s = - lim - lim — S n (44) 



/3^oo (3 n^o dn 



b+/3nq 



lim — lim-^-(ri — 1) / R(— w) dw + [ R(—w)dw 
3 ^oo p n ^o dn J J 



- log M(e, /) + r ^p(5\ + f (PP - 2e)(b + 0q) 

b 

lim \ [ R(-w) dw -^R(-b) + qbR'(-b) - \ [[ log V e »^}-^W 2 D 2 dP s (s). (45) 







Now, we use 1' Hospital's rule, re-substitute 6 and q, make use of 6 < oo and get (ITT]) . Note that for any 
bound on the amplitude of the signal set B, the parameter q is finite. Even without bound, q will remain 
finite for a well-defined minimization problem. The parameter b behaves in a more complicated manner. 
It can be both zero, finite, and infinite as j3 — > oo depending on the particular R-transform and the signal 
sets B s . For (3 £ (0, oo), the saddle-point limits have to be re-considered. 

V. Particular Results 

The general result leaves us with two components to specify: 1) The statistics of the random matrix 
entering the energy per symbol via its R-transform. 2) The relaxed signal alphabets B s Vs E S. While 
the relaxed alphabets characterize a particular method of precoding, the random matrix statistics depends 
on the wireless communication system. In the following, we will consider the following choice for the 
statistics of the random matrix. 

Consider a vector-valued communication system. Let the received vector be given as 

r = Ht + n (46) 

where n is white Gaussian noise. Let the components of the transmitted and received vectors be signals 
sent and received at different antenna elements, respectively. 

We want to ensure that the received signal is (up to additive noise) identical to the data vector. This 
design criteria leads us to choose the precoding matrix 

T = H*(HH*)~ 1 . (47) 

This means that we invert the channel and get r = x + n if the matrix inverse exists. This allows to 
keep the signal processing at the receiver at a minimum. This is advantageous if the receiver shall be a 
low-cost or battery-powered device. 

To model the statistics of the entries of H is a non-trivial task and a topic of ongoing research, see 
e.g. [2] and references therein. For sake of convenience, we choose in this first order approach that the 
entries of the channel matrix H are independent and identically distributed complex Gaussian random 
variables with zero mean and variance 1/N. For that case, we find in the appendix that 

, . 1 — a — \l (1 — a) 2 — Aaw 
R(w) = ^ '- (48) 

( 1 - a - a/(1 - a) 2 - Aaw) 

R'{w) = ± =J—. (49) 

Aaw 2 ^J{l — a) 2 — Aaw 
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It also turns out helpful to recognize that 



R 2 H = y/(l-cQ 2 -4 aw 
R'(w) a 

In the following, we compare the performances of several constructions of the redundant signal re- 
presentations for the channel model specified above. 

A. 1 -Dimensional Lattice 

Consider binary one-dimensional modulation. Let 

S = {0,1} (51) 

By = -B C K. (52) 

This is the mathematical description of binary phase- shift keying on the real line in context of vector 
precoding. 

Using the specifications above, we find in the limit j3 — > oo 



argmm 



argmm 



qR'( 


~b) 


1 2R 2 { 


-b) 


l qR'{- 


-b) 



X 



2 _ 2 2 

e~^~dz 



2R 2 (-b) 



ze 2 dz 
y/4nqR'(-b) 



(53) 
(54) 



Moreover, let without loss of generality — oo = c < c\ < ■ ■ ■ < cl < cl + i = +oo and 

Bi = {ci,c 2 , . . . ,c L } (55) 

This case describes Tomlinson-Harashima precoding [19], [7] with optimization over L different represen- 
tations for each information bit. An example of such a respresentation for integer lattice points is shown 
in Fig. [T] The boundary points of the Voronoi regions are 

Ci + Ci_i 



and the fixed-point equations for q and b become 

V2R(-b)v i+1 

1 , 

q 



(56) 



1 L 



2 — — i 

Ci e 2 az 



V2R(-b)vi 



y , \pi - c^i) Q 

i=2 

L 

(c{ - Ci_i) exp 

i=2 



R(-b)( Ci + c. 



i-lj 



y/2qR'{-b) 



AqR'{-b) 



^4nqR'(-b) 

with Q = exp(— x 2 /2)dx/\^2n denoting the Gaussian probability integral. 
For the case of no precoding at all, i.e. L = 1, we get 

b = 



q 



4 



clR(0). 



(57) 



(58) 



(59) 



(60) 
(61) 
(62) 



For the case of general L, we first restrict to the special case of a square channel matrix. The rectangular 
case is addressed subsequently. 
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1) Square Channel Matrix: For a — 1, ( |48l ) and ( 1491 respectively simplify to 

1 



Thus, we find 



R'(w) 
Ee — > oo 



1 

2{-w)l 



if lim 6 = 0. 



For positive values of b, we get 

9 = c i + W - c *-i) Q (kV^Q + 



i=2 
3 L 



^^(ci-Ci-Oexp 



i=2 



Vb(d + Q_i) 2 
2g 



and 



2v^ 



(63) 
(64) 

(65) 

(66) 
(67) 

(68) 



which makes the case distinction with respect to the asymptotic behavior of b obsolete. Moreover, we can 
combine the above 3 equations to find 



E s = 7l 



Cj +Cj-1 

/2~EI 



E ( c * - Ci_i)exp 

j=2 



(Cj+Ci-l) 2 

4B S 



(69) 



Numerical solutions to (l69l are shown in Table U for the equally spaced integer lattice 

So = {+1,-3, +5, -7, +9,...} 



(70) 



and various numbers of lattice points. Obviously, there is little improvement when going from two to 
three lattice points and negligible improvement for more than 3 lattice points. 

2) Rectangular Channel Matrix: For a rectangular channel matrix, the Gramian is only invertible for 
a < 1. However, the R-transform is well-defined for any positive aspect ratio. For singular random 
matrices, the R-transform reflects the fact that the asymptotic eigenvalue distribution has some point mass 
at infinity. 



Thus, we find 



({l-af+iaby (c l +c i . 1 ) 



/2qa 



i=2 



by-^Vi 1 -«) 2 + 4a6 ^ _vti_ Q) 2 +4aKci+Ci _ l) 2 

2 , ( c i - Ci-l) e 



a - 1 + a/(1 - a) 2 + 4a6 f =2 
It is convenient to replace the parameter b by the substitution 

p = y/(l - a) 2 + Aab 



Aqa. 



(71) 
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which gives 

/opA / P (q + q_i) 2 \ 

p = 1-at+W — > (q - Ci_i) exp 

V 7r ? ~^ V 4ga y 

and 

£ s = -. (72) 

p 

Finally, combining the last three equations, we get 

£ s = ^ (73) 

i=2 x ' 

The solutions of these fixed-point equations are shown by the solid lines in Fig. |2] Clearly for small 
load, the parameter q tends to 1, i.e. dB, as in that case, no gain due to precoding is possible and 
the symbol with smallest magnitude is preferred. The minimum of the transmit power is shown by the 
solid line in Fig. |3] Note that precoding enables to achieved finite transmitted energy per symbol even if 
the channel matrix is singular. This effect has already been explained for Marchenko-Pastur distributed 
random matrices. Unlike the curve without precoding, the curves for L > 1 do not have poles at the 
threshold load. Instead, a phase transition occurs and the energy per symbol jumps discontinuously from 
a finite value to infinity. In fact, it can be shown that the threshold load at which this happens is universal 
for a large class of random matrices in that it depends only on the specifics of the precoding lattice but 
not on the channel statistics. 



B. 2 -Dimensional Quadrature Lattice 

Consider now the case shown in Fig. |U It has the following properties: 

S = {00,01,10,11} 
B ly = -B* 0y Vye{0,l} 
B xl = +B* x0 Va;e{0,l} 



(74) 
(75) 
(76) 



This case extends the one-dimensional precoding of binary phase-shift keying (BPSK) on the real line to 
two-dimensional precoding of quaternary phase-shift keying (QPSK) in the complex plane such that Gray 
mapping is applied and we can consider the pre-coding for QPSK as independent pre-coding of BPSK in 
both quadrature components. 

The symmetry in both quadrature components implies that 



argmm 

c eK{Si +j } 



qR'(-b) 
2R 2 {-b) 



Z 2 dz 



argmm 

:e5ft{Bi +j } 



qR'(-b) 
2R 2 {-b) 



ze 



2 az 



y/nqR'(-b) 



(77) 



(78) 



Compared to the one-dimensional case, the only difference is that the right hand sides of the two fixed point 
equations are multiplied by a factor of 2 which stems from adding the contributions of both quadrature 
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Fig. 1. 2 one-dimensional equally spaced integer lattices representing the two binary states and 1, respectively. 



TABLE I 

Energy per symbol for inverted square channel. 



L 


1 


2 


3 


4 


CO 


E s 


CO 


2.6942 


2.6656 


2.6655 


2.6655 


E s [dB] 


CO 


4.3043 


4.2579 


4.2578 


4.2578 



CO 

cr 




Fig. 2. The macroscopic parameters q (upper lines), b (lower lines), and p (medium lines) versus the load a for L = 2, 3, 6, 100. shown 
by green, red, blue, and black lines, respectively. 
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Fig. 3. The transmitted energy per symbol versus the load for L — 1,2, 3, 6, 100 shown by the magenta, green, red, blue, and black lines, 
respectively. 



components. In order to allow for a fair comparison with 1 -dimensional modulation, we shall consider 
the energy per bit 

E h = (79) 
log 2 |o| 

to be the performance measure of choice. 

Due to de-coupling between quadrature components, we find that ( 1721) remains valid and p and q are 
given by 



(80) 



p=l-a + W ) (cj-Cj-ije 4 <*« . (81) 

V vrg ^— ' 



i=2 

The solutions to these fixed point equations are shown in Fig. [51 Remarkably, the energy per bit remains 

4 
3 



as small as E h = | for any load if L grows large. 



C. 2 -Dimensional Checkerboard Lattice 

Consider now the case shown in Fig. [6l This case extends the one-dimensional pre-coding of BPSK 
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on the real line to two-dimensional pre-coding of BPSK in the complex plane. Among others, it has the 
following properties: 

S = {0,1} (82) 
£i=j£ c C. (83) 

This mapping is like a checkerboard where the sets B\ and Bq correspond to the black and white fields, 
respectively. For this mapping, the boarderlines of the Voronoi regions are not parallel to the real and 
imaginary axes but intersect these by an angle of 45°. 

Considering an unconstrained lattice, i.e. infinitly many lattice points, we can rotate the lattice by 45° 
degrees without loss of generality due to the rotational invariance of the complex Gaussian integral kernel 
in the fixed-point equations for b and q. After rotation we find the same lattice as in the two-dimensional 
quadrature precoding except for a lattice scaling by a factor of 1/ \/2. Thus, the energy per symbol will 
be half the energy per symbol of quadrature precoding and the energy per bit will be identical. 



D. 2 -Dimensional Semi-Discrete Lattice 

Consider now the case shown in Fig. |7J This mapping is identical to the 1 -dimensional lattice except 
for the fact that the imaginary parts of the symbols in B x are left arbitrary. 
This mapping has the following properties 



By 



which lead to 

qR'(-b) 
Q 2R 2 {-b) 

b = -J— - 
2R{-b) 

For channel inversion, we have 



+ 



s 




= -B 


C 


argmin 









argmm 

ze5R{Bi} 



R'(-b) 



{0,1} 



gR'(-b) 
2R 2 {-b) 



gR'(-b) 
2R(-b) 2 



a 



2 2 dz 



x 



ze 



^7T 



*2 dz 



s/4xqR'{-bj 



(84) 
(85) 



(86) 



(87) 



(88) 



R 2 (-b) p' 

This enables us to easily solve the fixed point equations. 

Fig. M compares the complex semi-discrete lattice with the complex quadrature lattice in terms of energy 
per bit. Precoding with semi-discrete lattices achieves a remarkable gain which comes at the expense of 
reduced data rate. It is particularly worth to remark that the semi-discrete lattice with L = 1 outperforms 
all quadrature lattices for loads up to a ~ 0.479. Note that for L = 1, the sets B s are convex. Thus, 
the quadratic programming problem is convex since J is positive semidefinite and it can be solved with 
a polynomial time algorithm [1]. For large loads and large lattice size, the energy per bit approaches 
Eh = f • 



VI. Conclusions 

We have found that vector pre-coding can significantly reduce the required transmitted power. In 
fact, with appropriate pre-coding, the transmitted power stays always finite. Moreover, we found strong 
advantages of complex- valued pre-coding over real- valued pre-coding and a trade-off between data rate 
and required transmit power. 

We are aware of the fact that replica symmetry might not hold. Therefore, we have started investigating 
first order replica symmetry breaking (1RSB). The quantitative analysis is not finished yet, but qualitatively, 
the results remain unchanged for 1RSB. 
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Appendix 

Let px(x) be an arbitrary pdf such that both the Stieltjes transform defined in © and 

exist for some complex s with S(s) > 0. It can easily be checked that 

m x -i f~J = -s{l + sm x (s)). (90) 

Let s = m x l (—w). Then, we find 

m x -i (—^ J = -m x x {-w) (l - wm x \-w)) . (91) 

\m x (-w)J 



and 



1 



m 



-i 



m x \-w) x ~ 
With Definition 1, we find 



! (-m x \-w) (1 - wm x \-w))) . (92) 



R x -i (-wR x {w) (r x (w) + - ) 



Rx(w) + ± \ ^ J \ AV 1 wJJ wR x (w) {R x (w) + i) 

and 

Rx-i (~Rx(w) (1 + wR x (w))) . (94) 



R x (w 



It is well known [21], [20] that for aniVx aN random matrix H with i.i.d. entries of variance (aN) 1 , 
the R-transform of the limiting spectral measure P^dix) is given by 

R H f H (w) = T -^—. (95) 
1 — aw 

Letting X" 1 = H ] H, we find 

^(HtH)-l H = 1 + aR {tfH)-i ( W ) (l + wR (HlH)-^ ( W )) ( 96 ) 

with (|94l) . Solving (|96l) for the R-transform implies (|48l) . Note that for a > 1, the mean of the spectral 
measure is diverging. Thus, the R-transform must have a pole at w = which excludes the other solution 
of d963. 
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Fig. 4. 4 two-dimensional equally spaced integer quadrature lattices 



representing the four quaternary states 00, 01, 10, and 11, respectively. 
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Fig. 6. 2 two-dimensional equally spaced integer quadrature lattices representing the two binary states and 1, respectively. 
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Fig. 7. 2 two-dimensional equally spaced semi-discrete lattices representing the two binary states and 1, respectively. 



21 




Fig. 8. Energy per bit versus load for precoding with complex quadrature lattice (dashed lines) and semi-discrete lattice (solid lines) for 
L = 1, 2, 3, 6 shown by the magenta, green, red, and blue lines, respectively. 



